A Transformer-based representation-learning model with unified processing of multimodal input for clinical diagnostics (preprint)

During the diagnostic process, clinicians leverage multimodal information, such as chief complaints, medical images, and laboratory-test results. Deep-learning models for aiding diagnosis have yet to meet this requirement. Here we report a Transformer-based representation-learning model as a clinical diagnostic aid that processes multimodal input in a unified manner. Rather than learning modality-specific features, the model uses embedding layers to convert images and unstructured and structured text into visual tokens and text tokens, and bidirectional blocks with intramodal and intermodal attention to learn a holistic representation of radiographs, the unstructured chief complaint and clinical history, structured clinical information such as laboratory-test results and patient demographic information. The unified model outperformed an image-only model and non-unified multimodal diagnosis models in the identification of pulmonary diseases (by 12% and 9%, respectively) and in the prediction of adverse clinical outcomes in patients with COVID-19 (by 29% and 7%, respectively). Leveraging unified multimodal Transformer-based models may help streamline triage of patients and facilitate the clinical decision process.

A Self-Supervised Detail-Sensitive ViT-Based Model for COVID-19 X-ray Image Diagnosis: SDViT

COVID-19 has led to a severe impact on the society and healthcare system, with early diagnosis and effective treatment becoming critical. The Chest X-ray (CXR) is the most time-saving and cost-effective tool for diagnosing COVID-19. However, manual diagnosis through human eyes is time-consuming and tends to introduce errors. With the challenge of a large number of infections and a shortage of medical resources, a fast and accurate diagnosis technique is required. Manual detection is time-consuming, depends on individual experience, and tends to easily introduce errors. Deep learning methods can be used to develop automated detection and computer-aided diagnosis. However, they require a large amount of data, which is not practical due to the limited annotated CXR images. In this research, SDViT, an approach based on transformers, is proposed for COVID-19 diagnosis through image classification. We propose three innovations, namely, self-supervised learning, detail correction path (DCP), and domain transfer, then add them to the ViT Transformer architecture. Based on experimental results, our proposed method achieves an accuracy of 95.2381%, which is better performance compared to well-established methods on the X-ray Image dataset, along with the highest precision (0.952310), recall (0.963964), and F1-score (0.958102). Extensive experiments show that our model achieves the best performance on the synthetic-covid-cxr dataset as well. The experimental results demonstrate the advantages of our design for the classification task of COVID-19 X-ray images.

Effect analysis of meteorological conditions on air quality during the winter COVID-19 lockdown in Beijing

The influence of meteorological conditions on the pollution processes was investigated in this study by analyzing the changes of air quality as well as the characteristics of two persistent heavy pollution episodes during the Coronavirus Disea se 2019（COVID-19） prevention（January 24 to February 29） of 2020 winter compared with the same period of 2015～2019. Cold air intensity in 2020 winter was weaker with the cold surges frequency decreased by 50%. Air temperature was 0.73℃ higher, and wind speed and mixed layer height were 17.8% and 32.5% lower, respectively. Relative humidity and dew point temperature increased by 60. 9% and 48.1%, respectively. Northerly wind frequency reduced 7.5% while both of southerly and easterly wind increased 6.0%. As shown above, all meteorological conditions in 2020 winter were significantly more favorable for air pollution than the same historical period. Moreover, two heavy pollution episodes（January 24～29 and February 8～14） lasted for 59 and 75 hours were analyzed. At the cumulative stage, regional transport that can be divided into east and south channel greatly affected PM2.5, with the contribution of 70% and 58% for two episodes. By contrast, the contribution of local pollution was 67% and 48%, respectively, indicating the increased proportion of hygroscopic growth and secondary generation in the maintenance and aggravation stages. Under the meteorological background of "high humidity and high atmospheric stability", the combined effects of atmospheric vertical dynamics and horizontal convergence accumulated PM2.5 and water vapor in Beijing plain and prevented them from spreading beyond the boundary layer. Further bidirectional feedback between increased pollutants and meteorological factor s in stable boundary layer resulting in aggravation of pollution. According to EMI index, meteorological conditions during the epi demic prevention in 2020 winter caused an increase of 70.1% in PM2.5 concentration compared to pre-COVID-19. Emissions reduction caused by emergency measures for COVID-19 lockdown offset 53% of the adverse impact induced by meteorological conditions. As for the two episodes in 2020 winter, EMI was 26.9% and 19.7% larger than the average of other nine episodes in the correspond ing period of 2015～2019, and PM2.5 concentration was basically unchanged or slightly reduced. Overall, if the current social emission level is not changed, emission reduction caused by city blockade under special circumstances can only partially reduce the pollution concentration, however, cannot completely offset the adverse impact of meteorological conditions.

Clinically Applicable AI System for Accurate Diagnosis, Quantitative Measurements and Prognosis of COVID-19 Pneumonia Using Computed Tomography

Many COVID-19 patients infected by SARS-CoV-2 virus develop pneumonia (called novel coronavirus pneumonia, NCP) and rapidly progress to respiratory failure. However, rapid diagnosis and identification of high-risk patients for early intervention are challenging. Using a large computed Tomography (CT) database from 4,154 patients, we developed an AI system that can diagnose NCP and differentiate it from other common pneumonia and normal controls. The AI system can assist radiologists and physicians in performing a quick diagnosis especially when the health system is overloaded. Significantly, our AI system identified important clinical markers that correlated with the NCP lesion properties. Together with the clinical data, our AI system was able to provide accurate clinical prognosis that can aid clinicians to consider appropriate early clinical management and allocate resources appropriately. We have made this AI system available globally to assist the clinicians to combat COVID-19.